Skip to content

fix: make an auth failure a first-class outcome, not report text - #601

Merged
hyoshi merged 1 commit into
mainfrom
fix/daily-check-auth-failure
Aug 12, 2026
Merged

fix: make an auth failure a first-class outcome, not report text#601
hyoshi merged 1 commit into
mainfrom
fix/daily-check-auth-failure

Conversation

@hyoshi

@hyoshi hyoshi commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

The problem

A platform's credentials are missing or its token has expired. /daily-check runs to completion and ships a report that looks complete: the affected section carries error prose where numbers should be — "API error: Meta API request failed (status=400, ...)" or "Credentials not found. Set environment variable ..." — sitting next to real figures from the platforms that did answer, followed by the usual recommendations. A reader skimming it concludes the platform was quiet rather than unreadable.

The cause is that an auth failure arrives as an ordinary successful tool result whose text happens to be a sentence about credentials. _no_creds_result returns the sentence as the whole body; api_error_handler's blanket except Exception flattens an expired token into f"{API_ERROR_PREFIX} {exc}". Both are prose, so nothing downstream can tell "no spend" from "could not read" — and no read path consulted the one detector that exists (is_error_result, wired only to mutation/action-log gating).

The mechanism chosen

One machine-readable envelope, on every platform:

{"status": "auth_error", "auth_cause": "no_credentials", "detail": "Credentials not found. Set environment variable (META_ADS_ACCESS_TOKEN) or configure the credentials file."}

Why this shape:

  • It extends the convention already in the repo instead of inventing a second one. blind_spots / ChangeImportStatus (change-import) and DeliveryCollapseReport.status (no_credentials / data_unavailable) already answer the "what did I fail to see" question with a status field and a small vocabulary. A skill that has learned one has learned this.
  • JSON, not a text prefix. Tools return structured JSON data only is a design constraint, and the failure being fixed is precisely that prose was indistinguishable from prose. detail keeps the operator-facing sentence, so nothing readable is lost — it is a field now, never the whole answer.
  • Two causes, because their recovery differs. no_credentials = nothing configured (mureo configure / mureo auth setup); token_invalid = a credential exists and the platform rejected it (re-authorize). The issue noted that missing / expired / transient were all the same untyped string; transient stays an ordinary API error:.
  • The platform is not in the payload — the caller knows which platform it called, and adding it would mean the platform-neutral decorator guessing.

Produced in the two places every platform routes through, so Google Ads, Meta Ads and Search Console behave identically rather than by three separate edits:

Producer Cause Covers
_no_creds_result no_credentials all three native platforms (they all call it)
api_error_handler token_invalid HTTP 401/403 (Search Console), Google Ads authentication_error / authorization_error, PlatformAuthError

Meta needed one raise-site change: it answers an expired token with HTTP 400 and an OAuthException body, so the status code alone cannot tell a dead credential from a bad request. mureo/meta_ads/client.py now names that case (type == "OAuthException", or code 190 / 102, or a 401) and raises PlatformAuthError.

Classification is deliberately narrow, and unrecognized failures fall through to the existing API error: envelope. Over-claiming is the expensive direction: mislabelling a quota or validation error as an auth failure would send an operator to re-authorize a healthy account and withhold a report section that had perfectly good data behind it.

The skill branch

skills/daily-check/SKILL.md had no auth-failure branch — every failure-handling line in it addresses a different condition and says to keep going (the analytics-module fall-through, the official-hosted-MCP tool-surface fallbacks, step 2b's blind_spots). Two additions:

  • Step 4 — an auth_error result is a hole in the report, not data: never render detail where a metric belongs, never read that platform's missing figures as "quiet", carry the platform + auth_cause forward exactly as step 2b's blind_spots are carried, and keep going for every other platform.
  • Step 10 — if any platform came back auth_error, the report opens by saying it is partial, names each affected platform with its cause and recovery, and withholds every verdict, goal-progress line and recommendation that depends on that platform's numbers.

The #440 rule (never fail the whole daily-check because one platform's module errored) is untouched: the report degrades, it does not stop.

skills/_mureo-shared/SKILL.md documented an "Authentication Error" shape mureo never actually returned ({"error": "No credentials found..."}). It now documents the real envelope, so every skill that reads the shared prerequisite can see the marker.

Both copies of both skills are byte-identical (enforced by test_copies_are_byte_identical in the new suite and the existing one).

Not-obviously-related change that is load-bearing

is_error_result now recognises the auth envelope too. Without it, a mutation refused for a missing or rejected credential would stop matching the mutation gate and be promoted into action_log as a change that never happened — complete with an observation_due and a reversal plan for it. (A no_credentials mutation was already slipping past that gate before this PR, since the sentence never carried the API error: prefix.)

Corrections to the issue's trace

Verified each cited file:line against current main. Accurate: _handlers_meta_ads.py:37-41, _helpers.py:166-168 / :175 / :178-191 / :210-212, meta_ads/client.py:312-315, SKILL.md:59, SKILL.md:46, native_reversal.py:342-351 / :373, rollback/executor.py:135 / :185, exclusion_preflight.py:293-295, amazon_ads/bridge.py:202. Off (they point at an older revision of the file):

  • skills/daily-check/SKILL.md is 151 lines, not 143.
  • The official-hosted-MCP fallbacks are at :51/:52 (step 3) and :69/:70 (step 4), not :62/:63:62/:63 are the delivery-collapse block.
  • _no_creds_result returns its TextContent at :168, not :184 (:184 is inside is_error_result).
  • The server.py call sites are :1239 and :1260, not :1186 / :1207.

None of these change the diagnosis.

The other report skills

weekly-report and monthly-report have the same hole — neither mentions auth failure, credentials, or a partial report. Deliberately out of scope here: their report structures differ enough that it is not the same edit, and each needs its own pins. They do now inherit the marker's documentation via _mureo-shared, which they all read as a prerequisite. Worth a follow-up issue.

Also not touched: the reports.daily flag vocabulary in step 12 has no auth code, so a partial run's dashboard chip still has to go through {code: "custom"}. Adding a canonical code means touching the flag validator and the browser assets — separate change.

Tests

TDD, red first. New tests/test_auth_failure_envelope.py covers the vocabulary, the classifier (including the negative cases: 400/404/429/500, a Google Ads mutate_error, an ordinary RuntimeError), the envelope, is_auth_error_result, the is_error_result mutation gate, and api_error_handler routing. New tests/test_daily_check_auth_failure.py pins the skill branch in both copies. tests/test_meta_ads_client.py gains the Meta 400-vs-auth classification, including that an ordinary 400 stays a plain RuntimeError.

python -m pytest: 8893 passed, 12 failed. All 12 failures are the known local-environment set (locally-installed plugins leaking into tests that assume none — 513 tools vs 219 expected, smartnews_ads_* / line_ads_* in a frozenset() comparison — plus a locally-configured account scope): tests/analytics/builtin/test_live_clients.py (9), tests/test_mcp_server.py::TestListTools::test_list_tools_returns_all_tools, tests/test_mcp_server_plugin_wiring.py::test_no_plugins_is_additive_no_op, tests/test_mcp_tool_provider.py::test_default_discover_is_registry_and_yields_no_op_when_empty. None touch this change.

ruff check, black --check and ruff format --check clean on every file touched. mypy reports no error in mureo/core/auth_failure.py, mureo/mcp/_helpers.py or mureo/meta_ads/client.py (the 8 it does report are pre-existing missing-stub import-untyped errors in unrelated files).

Closes #580


Generated by Claude Code

When a platform's credentials were missing or its token had expired,
/daily-check ran to completion and shipped a report that looked
complete: the affected section carried error prose where numbers
belonged, next to real figures from the platforms that did answer,
followed by the usual recommendations. A reader skimming it concluded
the platform had been quiet rather than unreadable.

The cause was that an auth failure arrived as an ordinary *successful*
tool result whose text happened to be a sentence about credentials --
"Credentials not found. Set environment variable ..." from
_no_creds_result, or "API error: ..." from api_error_handler flattening
an expired token into the same untyped string as a quota error. Nothing
downstream could tell "no spend" from "could not read".

Auth failure now has one machine-readable envelope on every platform:
{"status": "auth_error", "auth_cause": "no_credentials" |
"token_invalid", "detail": ...}. It reuses the status-field convention
blind_spots and DeliveryCollapseReport already use for exactly this
"what did I fail to see" question rather than adding a second one, and
detail keeps the operator-facing sentence so nothing readable is lost.
The two causes are separated because their recovery differs: configure
the credential vs. re-authorize a rejected one.

It is produced in the two places every platform routes through, so
Google Ads, Meta Ads and Search Console behave identically:
_no_creds_result for a missing credential, and api_error_handler for a
rejected one -- an HTTP 401/403, a Google Ads authentication_error /
authorization_error, or a PlatformAuthError. Meta answers an expired
token with HTTP 400 and an OAuthException body, so the Meta client now
names that case explicitly instead of letting it flatten. The
classifier is deliberately narrow: an unrecognized failure stays an
ordinary "API error:", because mislabelling a quota or validation
failure would send an operator to re-authorize a healthy account and
would withhold a report section that had good data behind it.

/daily-check gained the branch it was missing. On an auth_error it
marks the report partial in its opening line, names each affected
platform with its cause and recovery, withholds every verdict,
goal-progress line and recommendation that depends on the missing
platform's data, and still completes for every platform that did
answer. The #440 rule -- never fail the whole daily-check because one
platform broke -- is untouched: the report degrades, it does not stop.
The shared _mureo-shared reference, which documented an envelope shape
mureo never actually returned, now documents the real one.

is_error_result recognises the new envelope too, so a mutation refused
for a missing or rejected credential is still kept out of action_log;
without that it would have been recorded as a change that never
happened, complete with an observation_due and a reversal plan.

Closes #580
@hyoshi
hyoshi force-pushed the fix/daily-check-auth-failure branch from bb88688 to f53aa5b Compare August 12, 2026 23:02
@hyoshi
hyoshi merged commit d3da447 into main Aug 12, 2026
13 checks passed
@hyoshi
hyoshi deleted the fix/daily-check-auth-failure branch August 12, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

/daily-check completes and ships a report when a platform's credentials fail

1 participant